Software with artificial intelligence-derived algorithms for analysing CT brain scans in people with a suspected acute stroke: a systematic review and cost-effectiveness analysis.

Background: Artificial intelligence-derived software technologies have been developed that are intended to facilitate the review of computed tomography brain scans in patients with suspected stroke. Objectives: To evaluate the clinical and cost-effectiveness of using artificial intelligence-derived software to support review of computed tomography brain scans in acute stroke in the National Health Service setting. Methods: Twenty-five databases were searched to July 2021. The review process included measures to minimise error and bias. Results were summarised by research question, artificial intelligence-derived software technology and study type. The health economic analysis focused on the addition of artificial intelligence-derived software-assisted review of computed tomography angiography brain scans for guiding mechanical thrombectomy treatment decisions for people with an ischaemic stroke. The de novo model (developed in R Shiny, R Foundation for Statistical Computing, Vienna, Austria) consisted of a decision tree (short-term) and a state transition model (long-term) to calculate the mean expected costs and quality-adjusted life-years for people with ischaemic stroke and suspected large-vessel occlusion comparing artificial intelligence-derived software-assisted review to usual care. Results: A total of 22 studies (30 publications) were included in the review; 18/22 studies concerned artificial intelligence-derived software for the interpretation of computed tomography angiography to detect large-vessel occlusion. No study evaluated an artificial intelligence-derived software technology used as specified in the inclusion criteria for this assessment. For artificial intelligence-derived software technology alone, sensitivity and specificity estimates for proximal anterior circulation large-vessel occlusion were 95.4% (95% confidence interval 92.7% to 97.1%) and 79.4% (95% confidence interval 75.8% to 82.6%) for Rapid (iSchemaView, Menlo Park, CA, USA) computed tomography angiography, 91.2% (95% confidence interval 77.0% to 97.0%) and 85.0 (95% confidence interval 64.0% to 94.8%) for Viz LVO (Viz.ai, Inc., San Fransisco, VA, USA) large-vessel occlusion, 83.8% (95% confidence interval 77.3% to 88.7%) and 95.7% (95% confidence interval 91.0% to 98.0%) for Brainomix (Brainomix Ltd, Oxford, UK) e-computed tomography angiography and 98.1% (95% confidence interval 94.5% to 99.3%) and 98.2% (95% confidence interval 95.5% to 99.3%) for Avicenna CINA (Avicenna AI, La Ciotat, France) large-vessel occlusion, based on one study each. These studies were not considered appropriate to inform cost-effectiveness modelling but formed the basis by which the accuracy of artificial intelligence plus human reader could be elicited by expert opinion. Probabilistic analyses based on the expert elicitation to inform the sensitivity of the diagnostic pathway indicated that the addition of artificial intelligence to detect large-vessel occlusion is potentially more effective (quality-adjusted life-year gain of 0.003), more costly (increased costs of £8.61) and cost-effective for willingness-to-pay thresholds of £3380 per quality-adjusted life-year and higher. Limitations and conclusions: The available evidence is not suitable to determine the clinical effectiveness of using artificial intelligence-derived software to support the review of computed tomography brain scans in acute stroke. The economic analyses did not provide evidence to prefer the artificial intelligence-derived software strategy over current clinical practice. However, results indicated that if the addition of artificial intelligence-derived software-assisted review for guiding mechanical thrombectomy treatment decisions increased the sensitivity of the diagnostic pathway (i.e. reduced the proportion of undetected large-vessel occlusions), this may be considered cost-effective. Future work: Large, preferably multicentre, studies are needed (for all artificial intelligence-derived software technologies) that evaluate these technologies as they would be implemented in clinical practice. Study registration: This study is registered as PROSPERO CRD42021269609. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR133836) and is published in full in Health Technology Assessment; Vol. 28, No. 11. See the NIHR Funding and Awards website for further award information.

Stroke is a serious life-threatening medical condition caused by a blood clot or haemorrhage in the brain. Quick and effective management, including a brain scan, of the patients with suspected stroke can make a big difference in their outcome. Artificial intelligence-derived computer programmes exist that are intended to help with the interpretation of computed tomography scans of the brain in stroke. We undertook a thorough review of the existing research into the effectiveness and value for money of using these programmes to help doctors and other specialists to interpret computed tomography brain scans. We found very little evidence to tell us how well artificial intelligence-derived computer programmes work in practice. Some studies have looked at artificial intelligence-derived computer programmes on their own (i.e. not taken together with a doctor's judgement, as they were designed to be used). Other studies have looked at what happens to patients who are treated for stroke when artificial intelligence-derived computer programmes are used; these studies provide no information about whether using artificial intelligence-derived computer programmes may have led to patients who could have benefitted from treatment being missed. It is unclear how well artificial intelligence-derived software-assisted review works when added to current clinical practice.

KardiaMobile 6L for measuring QT interval in people having antipsychotic medication to inform early value assessment: a systematic review.

Background: The indication for this assessment is the use of the KardiaMobile six-lead electrocardiogram device for the assessment of QT interval-based cardiac risk in service users prior to the initiation of, or for the monitoring of, antipsychotic medications, which are associated with an established risk of QT interval prolongation. Objectives: To provide an early value assessment of whether KardiaMobile six-lead has the potential to provide an effective and safe alternative to 12-lead electrocardiogram for initial assessment and monitoring of QT interval-based cardiac risk in people taking antipsychotic medications. Review methods: Twenty-seven databases were searched to April/May 2022. Review methods followed published guidelines. Where appropriate, study quality was assessed using appropriate risk of bias tools. Results were summarised by research question; accuracy/technical performance; clinical effects (on cardiac and psychiatric outcomes); service user acceptability/satisfaction; costs of KardiaMobile six-lead. Results: We did not identify any studies which provided information about the diagnostic accuracy of KardiaMobile six-lead, for the detection of corrected QT-interval prolongation, in any population. All studies which reported information about agreement between QT interval measurements (corrected and/or uncorrected) with KardiaMobile six-lead versus 12-lead electrocardiogram were conducted in non-psychiatric populations, used cardiologists and/or multiple readers to interpret electrocardiograms. Where reported or calculable, the mean difference in corrected QT interval between devices (12-lead electrocardiogram vs. KardiaMobile six-lead) was generally small (≤ 10 ms) and corrected QT interval measured using KardiaMobile six-lead was consistently lower than that measured using 12-lead electrocardiogram. All information about the use of KardiaMobile six-lead, in the context of QT interval-based cardiac risk assessment for service users who require antipsychotic medication, was taken from retrospective surveys of staff and service users who had chosen to use KardiaMobile six-lead during pilots, described in two unpublished project reports. It is important to note that both these project reports relate to pilot studies which were not intended to be used in wider evaluations of KardiaMobile six-lead for use in the NHS. Both reports included survey results which indicated that the use of KardiaMobile six-lead may be associated with reductions in the time taken to complete an electrocardiogram and costs, relative to 12-lead electrocardiogram, and that KardiaMobile six-lead was preferred over 12-lead electrocardiogram by almost all responding staff and service users. Limitations: There was a lack of published evidence about the efficacy of KardiaMobile six-lead for initial assessment and monitoring of QT interval-based cardiac risk in people taking antipsychotic medications. Conclusions: There is insufficient evidence to support a full diagnostic assessment evaluating the clinical and cost effectiveness of KardiaMobile six-lead, in the context of QT interval-based cardiac risk assessment for service users who require antipsychotic medication. The evidence to inform the aims of this early value assessment (i.e. to assess whether the device has the potential to be clinically effective and cost-effective) was also limited. This report includes a comprehensive list of research recommendations, both to reduce the uncertainty around this early value assessment and to provide the additional data needed to inform a full diagnostic assessment, including cost-effectiveness modelling. Study registration: This study is registered as PROSPERO CRD42022336695. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR135520) and is published in full in Health Technology Assessment; Vol. 28, No. 19. See the NIHR Funding and Awards website for further award information.

Some medicines used for people with certain mental health problems can increase the risk of developing serious heart conditions. Although these heart conditions are rare, it is generally recommended that people have an electrocardiogram examination before starting to take these medicines. People who need to continue these medications over a period of time may need additional electrocardiograms every so often, to check for any heart problems that have developed recently. KardiaMobile six-lead (or 6-lead) is a portable electrocardiogram that may offer a less intrusive way to take electrocardiogram measurements. This is because less undressing is needed as the electrodes are only applied to fingers of the left and right hand and the left ankle or knee and the cold gel is not needed. Testing using the KardiaMobile six-lead device can be carried out at the patient's home. These features might mean that the KardiaMobile six-lead device could be more acceptable than the 12-lead electrocardiogram to some patients. This assessment considered whether the KardiaMobile six-lead device has the potential to provide an effective and safe alternative to 12-lead electrocardiogram for initial assessment and monitoring of the risk of heart problems in people taking antipsychotic medications. Based on the available evidence, it remains unclear whether KardiaMobile six-lead has adequately demonstrated sufficient evidence of potential advantage(s) over current practice to justify further research to inform assessment of its clinical effectiveness and cost effectiveness. Our report provides detailed recommendations about the research needed, to provide further information about potential benefits so that a decision can be made about whether it should be used in the NHS in England, after further research has been completed.

The Parasite Reduction Ratio (PRR) Assay Version 2: Standardized Assessment of Plasmodium falciparum Viability after Antimalarial Treatment In Vitro.

With artemisinin-resistant Plasmodium falciparum parasites emerging in Africa, the need for new antimalarial chemotypes is persistently high. The ideal pharmacodynamic parameters of a candidate drug are a rapid onset of action and a fast rate of parasite killing or clearance. To determine these parameters, it is essential to discriminate viable from nonviable parasites, which is complicated by the fact that viable parasites can be metabolically inactive, whilst dying parasites can still be metabolically active and morphologically unaffected. Standard growth inhibition assays, read out via microscopy or [3H] hypoxanthine incorporation, cannot reliably discriminate between viable and nonviable parasites. Conversely, the in vitro parasite reduction ratio (PRR) assay is able to measure viable parasites with high sensitivity. It provides valuable pharmacodynamic parameters, such as PRR, 99.9% parasite clearance time (PCT99.9%) and lag phase. Here we report the development of the PRR assay version 2 (V2), which comes with a shorter assay duration, optimized quality controls and an objective, automated analysis pipeline that systematically estimates PRR, PCT99.9% and lag time and returns meaningful secondary parameters such as the maximal killing rate of a drug (Emax) at the assayed concentration. These parameters can be fed directly into pharmacokinetic/pharmacodynamic models, hence aiding and standardizing lead selection, optimization, and dose prediction.

Role of the faecal immunochemical test in patients with risk-stratified suspected colorectal cancer symptoms: A systematic review and meta-analysis to inform the ACPGBI/BSG guidelines.

Background: The UK National Institute for Health and Care Excellence (NICE), recommended in 2017 the use of the faecal immunochemical test (FIT) to guide investigations in patients presenting with NICE-defined low-risk symptoms suspicious for colorectal cancer (CRC). At that time, NICE did not recommend FIT use for high-risk symptoms. This is the first systematic review to evaluate the diagnostic accuracy of FIT in NICE-defined high and low-risk symptoms and was designed to inform the joint ACPGBI/BSG guidelines. Methods: We performed a systematic literature review and meta-analysis. PROSPERO registration number CRD42021224674. Medline and EMBASE databases were searched from inception to 31st March 2022. We included studies recruiting adult patients presenting with suspected CRC symptoms in whom FIT was performed and diagnostic accuracy data for CRC detection could be derived at a limit of detection (LoD) and/or 10 µg haemoglobin/gram faeces threshold in four commonly used analysers. FIT performance was assessed for high-risk, low-risk and individual symptoms where possible. Bivariate meta-analysis was performed where study numbers allowed. Findings: Thirty-one studies (79566 patients) met inclusion criteria. At 10 µg/g, for "all symptoms" (n = 35,945) sensitivity and specificity were 91.0% (95% CI: 88.9, 92.7) and 75.2% (95% CI: 69.6, 80.1); for "high-risk" symptoms (n = 18,264), 88.7% (95% CI: 84.4, 92.0) and 78.5% (95% CI: 73.0, 83.2); and for "low-risk" symptoms (n = 2161), 88.7% (95% CI: 78.1, 95.3) and 88.5% (95% CI: 87.1, 89.9), respectively. At LoD, for "all symptoms" (n = 26,056) sensitivity and specificity were 94.7% (95% CI: 90.5, 97.1) and 66.5% (95% CI: 58.7, 73.6); for "high-risk" symptoms (n = 16,768), 92.8% (95% CI: 86.4, 96.3) and 70.3% (95% CI: 66.5, 73.8); and for "low-risk" symptoms (n = 2082), 94.7% (95% CI: 85.4, 98.9) and 71.9% (95% CI: 69.9, 73.9), respectively. Summary estimates were similar across different analysers. Interpretation: FIT sensitivity for CRC detection is maximised at the LoD; its performance is similar in high and low-risk symptoms, and across different analysers where a common threshold is used. FIT performance for CRC detection is adequate and transferrable to clinical diagnostic pathways. Funding: This review was part-funded by NHS England awarded to RM Partners. RB and RC were funded by research fellowships awarded by Croydon University Hospital.

QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies.

Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.

High-sensitivity troponin assays for early rule-out of acute myocardial infarction in people with acute chest pain: a systematic review and economic evaluation.

BACKGROUND: Early diagnosis of acute myocardial infarction is important, but only 20% of emergency admissions for chest pain will actually have an acute myocardial infarction. High-sensitivity cardiac troponin assays may allow rapid rule out of myocardial infarction and avoid unnecessary hospital admissions. OBJECTIVES: To assess the clinical effectiveness and cost-effectiveness of high-sensitivity cardiac troponin assays for the management of adults presenting with acute chest pain, in particular for the early rule-out of acute myocardial infarction. METHODS: Sixteen databases were searched up to September 2019. Review methods followed published guidelines. Studies were assessed for quality using appropriate risk-of-bias tools. The bivariate model was used to estimate summary sensitivity and specificity for meta-analyses involving four or more studies; otherwise, random-effects logistic regression was used. The health economic analysis considered the long-term costs and quality-adjusted life-years associated with different troponin testing methods. The de novo model consisted of a decision tree and a state-transition cohort model. A lifetime time horizon (of 60 years) was used. RESULTS: Thirty-seven studies (123 publications) were included in the review. The high-sensitivity cardiac troponin test strategies evaluated are defined by the combination of four factors (i.e. assay, number and timing of tests, and threshold concentration), resulting in a large number of possible combinations. Clinical opinion indicated a minimum clinically acceptable sensitivity of 97%. When considering single test strategies, only those using a threshold at or near to the limit of detection for the assay, in a sample taken at presentation, met the minimum clinically acceptable sensitivity criterion. The majority of the multiple test strategies that met this criterion comprised an initial rule-out step, based on high-sensitivity cardiac troponin levels in a sample taken on presentation and a minimum symptom duration, and a second stage for patients not meeting the initial rule-out criteria, based on presentation levels of high-sensitivity cardiac troponin and absolute change after 1, 2 or 3 hours. Two large cluster randomised controlled trials found that implementation of an early rule-out pathway for myocardial infarction reduced length of stay and rate of hospital admission without increasing cardiac events. In the base-case analysis, standard troponin testing was both the most effective and the most costly. Other testing strategies with a sensitivity of 100% (subject to uncertainty) were almost equally effective, resulting in the same life-year and quality-adjusted life-year gain at up to four decimal places. Comparisons based on the next best alternative showed that for willingness-to-pay values below £8455 per quality-adjusted life-year, the Access High Sensitivity Troponin I (Beckman Coulter, Brea, CA, USA) [(symptoms > 3 hours AND < 4 ng/l at 0 hours) OR (< 5 ng/l AND Δ < 5 ng/l at 0 to 2 hours)] would be cost-effective. For thresholds between £8455 and £20,190 per quality-adjusted life-year, the Elecsys® Troponin-T high sensitive (Roche, Basel, Switzerland) (< 12 ng/l at 0 hours AND Δ < 3 ng/l at 0 to 1 hours) would be cost-effective. For a threshold > £20,190 per quality-adjusted life-year, the Dimension Vista® High-Sensitivity Troponin I (Siemens Healthcare, Erlangen, Germany) (< 5 ng/l at 0 hours AND Δ < 2 ng/l at 0 to 1 hours) would be cost-effective. CONCLUSIONS: High-sensitivity cardiac troponin testing may be cost-effective compared with standard troponin testing. STUDY REGISTRATION: This study is registered as PROSPERO CRD42019154716. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Evidence Synthesis programme and will be published in full in Health Technology Assessment; Vol. 25, No. 33. See the NIHR Journals Library website for further project information.

Heart disease is a leading cause of death in the UK, with myocardial infarction (heart attack) accounting for approximately 4% of all deaths recorded in 2018. Many people attend hospital with chest pain and suspected myocardial infarction, and chest pain has been reported as the most common cause of hospital admissions in the UK, accounting for approximately 5% of all emergency admissions in 2017­18. It is important to diagnose people who are suspected of having a myocardial infarction as early as possible to ensure quick and effective treatment. However, only around 20% of emergency admissions for chest pain will actually have an myocardial infarction and there are many other possible causes of chest pain (e.g. gastro-oesophageal disorders, muscle pain, anxiety or stable ischaemic heart disease). Current practice for ruling out myocardial infarction includes blood tests taken when the patient is first seen in the emergency department and repeated after 3­6 hours or 10­12 hours, depending on the test used. Tests that can quickly tell which patients do not have myocardial infarction could therefore avoid unnecessary hospital admissions and anxiety for many people. We aimed to assess the clinical effectiveness and cost-effectiveness of high-sensitivity troponin tests, used as single tests or repeated over a short time, for the early rule out of myocardial infarction in people who present to hospital with chest pain. We found that high-sensitivity troponin tests can safely rule out myocardial infarction within the 4-hour NHS emergency department target. Health economic analyses indicated that high-sensitivity tests may be considered value for money compared with standard troponin tests, which require repeat testing at 10­12 hours.

Optimizing the Use of High-Sensitivity Troponin Assays for the Early Rule-out of Myocardial Infarction in Patients Presenting with Chest Pain: A Systematic Review.

BACKGROUND: We assessed the accuracy and clinical effectiveness of high-sensitivity cardiac troponin (hs-cTn) assays for early rule-out of non-ST-segment elevation myocardial infarction (NSTEMI) in adults presenting with acute chest pain. METHODS: Sixteen databases were searched to September 2019. Review methods followed published guidelines. The bivariate model was used to estimate summary sensitivity and specificity with 95% confidence intervals for meta-analyses involving 4 or more studies, otherwise random-effects logistic regression was used. RESULTS: Thirty-seven studies (124 publications) were included in the review. The hs-cTn test strategies evaluated in the included studies were defined by the combination of 4 factors (assay, number of tests, timing of tests, and threshold concentration or change in concentration between tests). Clinical opinion indicated a minimum acceptable sensitivity of 97%. A single test at presentation using a threshold at or near the assay limit of detection could reliably rule-out NSTEMI for a range of hs-cTn assays. Serial testing strategies, which include an immediate rule-out step, increased the proportion ruled out without loss of sensitivity. Finally, serial testing strategies without an immediate rule-out step had excellent sensitivity and specificity, but at the expense of the option for immediate patient discharge. CONCLUSION: Test strategies that comprise an initial rule-out step, based on low hs-cTn concentrations at presentation and a minimum symptom duration, and a second step for those not ruled-out that incorporates a small absolute change in hs-cTn at 1, 2, or 3 hours, produce the highest rule-out rates with a very low risk of missed NSTEMI. PROSPERO REGISTRATION: CRD42019154716.

Cannabidiol for Adjuvant Treatment of Seizures Associated with Lennox-Gastaut Syndrome and Dravet Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

GW Research Ltd. provided two separate, but similar, submissions to the National Institute for Health and Care Excellence (NICE) on the clinical and cost-effectiveness of cannabidiol (CBD) 10 mg/kg/day, trade name Epidyolex®, for the adjunctive treatment of seizures associated with Lennox-Gastaut syndrome (LGS) and Dravet syndrome (DS). This paper highlights important methodological issues related to the company submissions, the Evidence Review Group (ERG) reports, and the subsequent development of the NICE guidance by the Appraisal Committee (AC) for the use of CBD. The company identified four randomised controlled trials (RCTs) of CBD (GWPCARE1 and GWPCARE2 for DS, and GWPCARE3 and GWPCARE4 for LGS) and an ongoing open-label extension study (GWPCARE5) as relevant to both submissions. In these RCTs, CBD in addition to current clinical management (CCM) was compared to CCM without CBD (i.e. CCM plus placebo). GWPCARE2 and GWPCARE3 were three-arm studies and compared two dosages of CBD (10 mg/kg/day and 20 mg/kg/day) in addition to CCM and CCM plus placebo. GWPCARE1 and GWPCARE4 compared CBD (20 mg/kg/day) in addition to CCM and CCM plus placebo. Both DS patients in GWPCARE2 and LGS patients in GWPCARE3 who received 10 mg/kg/day CBD in addition to CCM achieved better seizure frequency outcomes than those who received CCM plus placebo. In the company's base case, use of CBD for LGS patients resulted in an incremental cost-effectiveness ratio (ICER) of £31,107 per quality-adjusted life year (QALY) gained and, for DS patients, £36,046 per QALY gained versus CCM. The ERG considered that these ICERs were extremely uncertain and suffered from validity issues concerning model structure (e.g. patients receiving CCM moved back to baseline drop seizure frequency), input (e.g. inclusion of caregivers' QALYs), and transparency issues (e.g. hidden worksheets and coding in Visual Basic for Applications), and hence incorporated adjustments to the original base case which increased the ICERs. During the process, the European Medicines Agency (EMA) licence granted marketing authorisation for CBD only in conjunction with clobazam. Hence, the company provided evidence from this subgroup in an additional submission, which resulted in an ICER of £33,721 per QALY gained for LGS and an ICER of £32,471 per QALY gained for DS. In this submission and clarifications, the ERG was able to verify and validate most of the company's responses to the ERG's concerns. However, some issues remained regarding the face validity of model assumptions on patient pathways after treatment discontinuation. Finally, the AC recommended CBD with clobazam as an option for treating seizures associated with LGS and DS in patients aged 2 years and older only if (1) the frequency of drop seizures is checked every 6 months and CBD is stopped if the frequency has not fallen by at least 30% compared with 6 months before starting treatment and (2) the company provides CBD according to the commercial arrangement.

Author Correction: Chimeric peptidomimetic antibiotics against Gram-negative bacteria.

An Amendment to this paper has been published and can be accessed via a link at the top of the paper.

Chimeric peptidomimetic antibiotics against Gram-negative bacteria.

There is an urgent need for new antibiotics against Gram-negative pathogens that are resistant to carbapenem and third-generation cephalosporins, against which antibiotics of last resort have lost most of their efficacy. Here we describe a class of synthetic antibiotics inspired by scaffolds derived from natural products. These chimeric antibiotics contain a ß-hairpin peptide macrocycle linked to the macrocycle found in the polymyxin and colistin family of natural products. They are bactericidal and have a mechanism of action that involves binding to both lipopolysaccharide and the main component (BamA) of the ß-barrel folding complex (BAM) that is required for the folding and insertion of ß-barrel proteins into the outer membrane of Gram-negative bacteria. Extensively optimized derivatives show potent activity against multidrug-resistant pathogens, including all of the Gram-negative members of the ESKAPE pathogens1. These derivatives also show favourable drug properties and overcome colistin resistance, both in vitro and in vivo. The lead candidate is currently in preclinical toxicology studies that-if successful-will allow progress into clinical studies that have the potential to address life-threatening infections by the Gram-negative pathogens, and thus to resolve a considerable unmet medical need.

The sensitivity and specificity of the mannitol bronchial challenge test to identify asthma in different populations: a systematic review.

OBJECTIVE: Asthma is associated with bronchial hyperresponsiveness, assessed by bronchial provocation tests such as the mannitol test. We aimed to assess the data on sensitivity and specificity of the mannitol test in diagnosing asthma. DATA SOURCES: We searched electronically the Medline, Embase and Central databases from 1997 to 2019. STUDY SELECTION: Inclusion criteria were the assessment of the validity of the mannitol test. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2). Data were extracted according to a prespecified list and analysed qualitatively. RESULTS: A total of 27 studies (4589 individuals, age 6–85 years, cross-sectional [n = 18] and case-controlled [n = 9] study design) were included. Overall sensitivity and specificity ranged from 8% (95% confidence interval [CI] 1–27) to 100% (95% CI 93–100) and 75% (95% CI 67–82) to 100% (95% CI 85–100). Excluding case-controlled design, studies conducted in a clinical setting showed a range from 19% (95% CI 14–27) to 91% (95% CI 59–100) for sensitivity and from 75% (95% CI 67–82) to 100% (95% CI 80–100) for specificity. Heterogeneity was high owing to differences in the populations examined and the methods used. CONCLUSIONS: Studies on the accuracy of the mannitol test were heterogeneous. Overall specificity was higher than sensitivity and therefore the mannitol test seems to be a suitable diagnostic tool to confirm asthma. However, the high level of heterogeneity among the included studies makes a conclusive statement on the accuracy of the mannitol test difficult and further research is needed. As bronchial provocation tests can be especially useful in patients with an intermediate probability of asthma diagnosis, further studies are needed that include subjects with asthma symptoms but intermediate probability of asthma diagnosis.

PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies.

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration.

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Clinically inappropriate post hoc exclusion of study participants from test accuracy calculations: the ROMA score, an example from a recent NICE diagnostic assessment.

OBJECTIVE: To explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies. METHODS: We used data from a systematic review, conducted for a NICE diagnostic assessment of risk scores to inform secondary care decisions about specialist referral for women with suspected ovarian cancer, to explore how the definition of the target condition and post hoc exclusion of participants can limit the usefulness of diagnostic accuracy studies to inform clinical practice. RESULTS: Fourteen of the studies evaluated the ROMA score, nine used Abbott ARCHITECT tumour marker assays, five used Roche Elecsys. The summary sensitivity estimate (Abbott ARCHITECT) was highest, 95.1% (95% CI: 92.4 to 97.1%), where analyses excluded participants with borderline tumours or malignancies other than epithelial ovarian cancer and lowest, 75.0% (95% CI: 60.4 to 86.4%), where all participants were included. Results were similar for Roche Elecsys tumour marker assays. Although the number of patients involved was small, data from studies that reported diagnostic accuracy for both the whole study population and with post hoc exclusion of those with borderline or non-epithelial malignancies suggested that patients with borderline or malignancies other than epithelial ovarian cancer accounts for between 50 and 85% of false-negative ROMA scores. CONCLUSIONS: Our results illustrate the potential consequences of inappropriate population selection in diagnostic studies; women with non-epithelial ovarian cancers or non-ovarian primaries, and those borderline tumours may be disproportionately represented among those with false negative, 'low risk' ROMA scores. These observations highlight the importance of giving careful consideration to how the target condition has been defined when assessing whether the diagnostic accuracy estimates reported in clinical studies will translate into clinical utility in real-world settings.

Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis.

BACKGROUND: Ovarian cancer is the sixth most common cancer in UK women and can be difficult to diagnose, particularly in the early stages. Risk-scoring can help to guide referral to specialist centres. OBJECTIVES: To assess the clinical and cost-effectiveness of risk scores to guide referral decisions for women with suspected ovarian cancer in secondary care. METHODS: Twenty-one databases, including MEDLINE and EMBASE, were searched from inception to November 2016. Review methods followed published guidelines. The meta-analysis using weighted averages and random-effects modelling was used to estimate summary sensitivity and specificity with 95% confidence intervals (CIs). The cost-effectiveness analysis considered the long-term costs and quality-adjusted life-years (QALYs) associated with different risk-scoring methods, and subsequent care pathways. Modelling comprised a decision tree and a Markov model. The decision tree was used to model short-term outcomes and the Markov model was used to estimate the long-term costs and QALYs associated with treatment and progression. RESULTS: Fifty-one diagnostic cohort studies were included in the systematic review. The Risk of Ovarian Malignancy Algorithm (ROMA) score did not offer any advantage over the Risk of Malignancy Index 1 (RMI 1). Patients with borderline tumours or non-ovarian primaries appeared to account for disproportionately high numbers of false-negative, low-risk ROMA scores. (Confidential information has been removed.) To achieve similar levels of sensitivity to the Assessment of Different NEoplasias in the adneXa (ADNEX) model and the International Ovarian Tumour Analysis (IOTA) group's simple ultrasound rules, a very low RMI 1 decision threshold (25) would be needed; the summary sensitivity and specificity estimates for the RMI 1 at this threshold were 94.9% (95% CI 91.5% to 97.2%) and 51.1% (95% CI 47.0% to 55.2%), respectively. In the base-case analysis, RMI 1 (threshold of 250) was the least effective [16.926 life-years (LYs), 13.820 QALYs] and the second cheapest (£5669). The IOTA group's simple ultrasound rules (inconclusive, assumed to be malignant) were the cheapest (£5667) and the second most effective [16.954 LYs, 13.841 QALYs], dominating RMI 1. The ADNEX model (threshold of 10%), costing £5699, was the most effective (16.957 LYs, 13.843 QALYs), and compared with the IOTA group's simple ultrasound rules, resulted in an incremental cost-effectiveness ratio of £15,304 per QALY gained. At thresholds of up to £15,304 per QALY gained, the IOTA group's simple ultrasound rules are cost-effective; the ADNEX model (threshold of 10%) is cost-effective for higher thresholds. LIMITATIONS: Information on the downstream clinical consequences of risk-scoring was limited. CONCLUSIONS: Both the ADNEX model and the IOTA group's simple ultrasound rules may offer increased sensitivity relative to current practice (RMI 1); that is, more women with malignant tumours would be referred to a specialist multidisciplinary team, although more women with benign tumours would also be referred. The cost-effectiveness model supports prioritisation of sensitivity over specificity. Further research is needed on the clinical consequences of risk-scoring. STUDY REGISTRATION: This study is registered as PROSPERO CRD42016053326. FUNDING DETAILS: The National Institute for Health Research Health Technology Assessment programme.

Faecal immunochemical tests (FIT) can help to rule out colorectal cancer in patients presenting in primary care with lower abdominal symptoms: a systematic review conducted to inform new NICE DG30 diagnostic guidance.

BACKGROUND: This study has attempted to assess the effectiveness of quantitative faecal immunochemical tests (FIT) for triage of people presenting with lower abdominal symptoms, where a referral to secondary care for investigation of suspected colorectal cancer (CRC) is being considered, particularly when the 2-week criteria are not met. METHODS: We conducted a systematic review following published guidelines for systematic reviews of diagnostic tests. Twenty-one resources were searched up until March 2016. Summary estimates were calculated using a bivariate model or a random-effects logistic regression model. RESULTS: Nine studies are included in this review. One additional study, included in our systematic review, was provided as 'academic in confidence' and cannot be described herein. When FIT was based on a single faecal sample and a cut-off of 10 µg Hb/g faeces, sensitivity estimates indicated that a negative result using either the OC-Sensor or HM-JACKarc may be adequate to rule out nearly all CRC; the summary estimate of sensitivity for the OC-Sensor was 92.1% (95% confidence interval, CI 86.9-95.3%), based on four studies (n = 4091 participants, 176 with CRC), and the only study of HM-JACKarc to assess the 10 µg Hb/g faeces cut-off (n = 507 participants, 11 with CRC) reported a sensitivity of 100% (95% CI 71.5-100%). The corresponding specificity estimates were 85.8% (95% CI 78.3-91.0%) and 76.6% (95% CI 72.6-80.3%), respectively. When the diagnostic criterion was changed to include lower grades of neoplasia, i.e. the target condition included higher risk adenoma (HRA) as well as CRC, the rule-out performance of both FIT assays was reduced. CONCLUSIONS: There is evidence to suggest that triage using FIT at a cut-off around 10 µg Hb/g faeces has the potential to correctly rule out CRC and avoid colonoscopy in 75-80% of symptomatic patients. SYSTEMATIC REVIEW REGISTRATION: PROSPERO 42016037723.

Faecal immunochemical tests to triage patients with lower abdominal symptoms for suspected colorectal cancer referrals in primary care: a systematic review and cost-effectiveness analysis.

BACKGROUND: Colorectal cancer (CRC) is the third most common cancer in the UK. Presenting symptoms that can be associated with CRC usually have another explanation. Faecal immunochemical tests (FITs) detect blood that is not visible to the naked eye and may help to select patients who are likely to benefit from further investigation. OBJECTIVES: To assess the effectiveness of FITs [OC-Sensor (Eiken Chemical Co./MAST Diagnostics, Tokyo, Japan), HM-JACKarc (Kyowa Medex/Alpha Laboratories Ltd, Tokyo, Japan), FOB Gold (Sentinel/Sysmex, Sentinel Diagnostics, Milan, Italy), RIDASCREEN Hb or RIDASCREEN Hb/Hp complex (R-Biopharm, Darmstadt, Germany)] for primary care triage of people with low-risk symptoms. METHODS: Twenty-four resources were searched to March 2016. Review methods followed published guidelines. Summary estimates were calculated using a bivariate model or a random-effects logistic regression model. The cost-effectiveness analysis considered long-term costs and quality-adjusted life-years (QALYs) that were associated with different faecal occult blood tests and direct colonoscopy referral. Modelling comprised a diagnostic decision model, a Markov model for long-term costs and QALYs that were associated with CRC treatment and progression, and a Markov model for QALYs that were associated with no CRC. RESULTS: We included 10 studies. Using a single sample and 10 µg Hb/g faeces threshold, sensitivity estimates for OC-Sensor [92.1%, 95% confidence interval (CI) 86.9% to 95.3%] and HM-JACKarc (100%, 95% CI 71.5% to 100%) indicated that both may be useful to rule out CRC. Specificity estimates were 85.8% (95% CI 78.3% to 91.0%) and 76.6% (95% CI 72.6% to 80.3%). Triage using FITs could rule out CRC and avoid colonoscopy in approximately 75% of symptomatic patients. Data from our systematic review suggest that 22.5-93% of patients with a positive FIT and no CRC have other significant bowel pathologies. The results of the base-case analysis suggested minimal difference in QALYs between all of the strategies; no triage (referral straight to colonoscopy) is the most expensive. Faecal immunochemical testing was cost-effective (cheaper and more, or only slightly less, effective) compared with no triage. Faecal immunochemical testing was more effective and costly than guaiac faecal occult blood testing, but remained cost-effective at a threshold incremental cost-effectiveness ratio of £30,000. The results of scenario analyses did not differ substantively from the base-case. Results were better for faecal immunochemical testing when accuracy of the guaiac faecal occult blood test (gFOBT) was based on studies that were more representative of the correct population. LIMITATIONS: Only one included study evaluated faecal immunochemical testing in primary care; however, all of the other studies evaluated faecal immunochemical testing at the point of referral. Further, validation data for the Faecal haemoglobin, Age and Sex Test (FAST) score, which includes faecal immunochemical testing, showed no significant difference in performance between primary and secondary care. There were insufficient data to adequately assess FOB Gold, RIDASCREEN Hb or RIDASCREEN Hb/Hp complex. No study compared FIT assays, or FIT assays versus gFOBT; all of the data included in this assessment refer to the clinical effectiveness of individual FIT methods and not their comparative effectiveness. CONCLUSIONS: Faecal immunochemical testing is likely to be a clinically effective and cost-effective strategy for triaging people who are presenting, in primary care settings, with lower abdominal symptoms and who are at low risk for CRC. Further research is required to confirm the effectiveness of faecal immunochemical testing in primary care practice and to compare the performance of different FIT assays. STUDY REGISTRATION: This study is registered as PROSPERO CRD42016037723. FUNDING: The National Institute for Health Research Health Technology Assessment programme.

Erratum to: Methods for evaluating medical tests and biomarkers.

[This corrects the article DOI: 10.1186/s41512-016-0001-y.].

ImmunoCAP® ISAC and Microtest for multiplex allergen testing in people with difficult to manage allergic disease: a systematic review and cost analysis.

BACKGROUND: Allergy is a form of immune-mediated exaggerated sensitivity (hypersensitivity) to a substance that is either inhaled, swallowed, injected or comes into contact with the skin. Foreign substances that provoke allergies are called allergens. It has been claimed that multiplex allergen testing may help in diagnosing the cause of symptoms in patients with an unclear cause of allergy or who are allergic to more than one substance. OBJECTIVES: To evaluate multiplex allergen testing [devices that can measure the presence of multiple immunoglobulin E (IgE) antibodies in a patient's blood at the same time], by assessing (1) clinical effectiveness (allergy symptoms, incidence of acute exacerbations, mortality, adverse events of testing and treatment, health-care presentations or admissions, health-related quality of life); (2) effects on treatment (diet, immunotherapy medications, other potential testing); (3) any additional diagnostic information provided by multiplex allergen testing; and (4) cost-effectiveness (cost of different assessment strategies). METHODS: Fifteen databases were searched from 2005 to April 2015, including MEDLINE (via OvidSp), MEDLINE In-Process Citations, MEDLINE Daily Update, PubMed (National Library of Medicine), EMBASE, Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials (CENTRAL), Database of Abstracts of Reviews of Effects (DARE), Health Technology Assessment (HTA) database, Science Citation Index (SCI), Conference Proceedings Citation Index-Science (CPCI-S), BIOSIS Previews, Latin American and Caribbean Health Sciences Literature (LILACS), National Institute for Health Research (NIHR) HTA programme, and the US Food and Drug Administration (FDA); supplementary searches of conference proceedings and trials registries were performed. Review methods followed published guidance from the Cochrane Collaboration and the Centre for Reviews and Dissemination, University of York, UK. The methodological quality of included studies was assessed using appropriate published tools or a review-specific tool designed by the project team. Studies were summarised in a narrative synthesis. Owing to a lack of data on the clinical effectiveness of multiplex allergen testing, no long-term cost-effectiveness model was developed. A conceptual model structure was developed and cost analyses were performed to examine the short-term costs of various possible diagnostic pathways. RESULTS: Fifteen studies were included in the review. The very limited available data indicated that the addition of multiplex allergen testing [ImmunoCAP(®) Immuno Solid-phase Allergen Chip (ISAC), Thermo Fisher Scientific/Phadia AB, Uppsala, Sweden] to standard diagnostic work-up can change the clinicians' views on the diagnosis, management and treatment of patients. There was some indication that the use of ImmunoCAP ISAC testing may be useful to guide decisions on the discontinuation of restrictive diets, the content of allergen-specific immunotherapy (SIT) prescriptions, and whether or not patients should receive SIT. However, none of the studies that we identified reported any information on clinical outcomes subsequent to changes in treatment or management. There was some evidence that ImmunoCAP ISAC may be useful for discriminating allergens that are structurally similar and are recognised by the same IgE antibody (cross-immunoreactive). No data were available for Microtest (Microtest Matrices Ltd, London, UK). Detailed cost analyses suggested that multiplex allergen testing would have to result in a substantial reduction of the proportions of patients receiving single IgE testing and oral food challenge tests in order to be cost-saving in the short term. CONCLUSIONS: No recommendations for service provision can be made based on the analyses included in this report. It is suggested that a consensus-based protocol for the use of multiplex allergen testing be developed. The clinical effectiveness and cost-effectiveness of the proposed protocol should then be assessed by comparing long-term clinical and quality of life outcomes and resource use in patients managed using the protocol with those managed using a standard diagnostic pathway. STUDY REGISTRATION: This study is registered as PROSPERO CRD42015019739. FUNDING: This project was a Diagnostic Assessment Report commissioned by the NIHR HTA programme on behalf of the National Institute for Health and Care Excellence.

Procalcitonin testing to guide antibiotic therapy for the treatment of sepsis in intensive care settings and for suspected bacterial infection in emergency department settings: a systematic review and cost-effectiveness analysis.

BACKGROUND: Determination of the presence or absence of bacterial infection is important to guide appropriate therapy and reduce antibiotic exposure. Procalcitonin (PCT) is an inflammatory marker that has been suggested as a marker for bacterial infection. OBJECTIVES: To assess the clinical effectiveness and cost-effectiveness of adding PCT testing to the information used to guide antibiotic therapy in adults and children (1) with confirmed or highly suspected sepsis in intensive care and (2) presenting to the emergency department (ED) with suspected bacterial infection. METHODS: Twelve databases were searched to June 2014. Randomised controlled trials were assessed for quality using the Cochrane Risk of Bias tool. Summary relative risks (RRs) and weighted mean differences (WMDs) were estimated using random-effects models. Heterogeneity was assessed visually using forest plots and statistically using the I (2) and Q statistics and investigated through subgroup analysis. The cost-effectiveness of PCT testing in addition to current clinical practice was compared with current clinical practice using a decision tree with a 6 months' time horizon. RESULTS: Eighteen studies (36 reports) were included in the systematic review. PCT algorithms were associated with reduced antibiotic duration [WMD -3.19 days, 95% confidence interval (CI) -5.44 to -0.95 days, I (2) = 95.2%; four studies], hospital stay (WMD -3.85 days, 95% CI -6.78 to -0.92 days, I (2) = 75.2%; four studies) and a trend towards reduced intensive care unit (ICU) stay (WMD -2.03 days, 95% CI -4.19 to 0.13 days, I (2) = 81.0%; four studies). There were no differences for adverse clinical outcomes. PCT algorithms were associated with a reduction in the proportion of adults (RR 0.77, 95% CI 0.68 to 0.87; seven studies) and children (RR 0.86, 95% CI 0.80 to 0.93) receiving antibiotics, reduced antibiotic duration (two studies). There were no differences for adverse clinical outcomes. All but one of the studies in the ED were conducted in people presenting with respiratory symptoms. Cost-effectiveness: the base-case analyses indicated that PCT testing was cost-saving for (1) adults with confirmed or highly suspected sepsis in an ICU setting; (2) adults with suspected bacterial infection presenting to the ED; and (3) children with suspected bacterial infection presenting to the ED. Cost-savings ranged from £368 to £3268. Moreover, PCT-guided treatment resulted in a small quality-adjusted life-year (QALY) gain (ranging between < 0.001 and 0.005). Cost-effectiveness acceptability curves showed that PCT-guided treatment has a probability of ≥ 84% of being cost-effective for all settings and populations considered (at willingness-to-pay thresholds of £20,000 and £30,000 per QALY). CONCLUSIONS: The limited available data suggest that PCT testing may be effective and cost-effective when used to guide discontinuation of antibiotics in adults being treated for suspected or confirmed sepsis in ICU settings and initiation of antibiotics in adults presenting to the ED with respiratory symptoms and suspected bacterial infection. However, it is not clear that observed costs and effects are directly attributable to PCT testing, are generalisable outside people presenting with respiratory symptoms (for the ED setting) and would be reproducible in the UK NHS. Further studies are needed to assess the effectiveness of adding PCT algorithms to the information used to guide antibiotic treatment in children with suspected or confirmed sepsis in ICU settings. Additional research is needed to examine whether the outcomes presented in this report are fully generalisable to the UK. STUDY REGISTRATION: This study is registered as PROSPERO CRD42014010822. FUNDING: The National Institute for Health Research Health Technology Assessment programme.